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1 INTRODUCTION 


1 Introduction 


This paper gives a brief overview of game 
theory. Therefore in the first section I 
want to outline what game theory gener- 
ally is and where it is applied. 

In the next section, I introduce some 
of the most important terms used in game 
theory, such as normal form games and 
Nash equilibrium as well as some of the 
most popular games, e.g. the Prisoner’s 
Dilemma or the Ultimatum Game. 

I then present the basic concepts of 
evolutionary game theory (EGT), a more 
specialized branch of game theory. We 
will see how EGT uses new concepts such 
as evolutionary stable strategies (ESS) and 
replicator dynamics, and its importance 
to sciences like biology and physics. 

At last I present two examples, pro- 
grammed in the language NetLogo, which 
will demonstrate the applications of EGT 
and the similarities to condensed matter 
physics. 


1.1 Game Theory — What is it? 


The concepts of game theory provide a 
common language to formulate, structure, 
analyse and eventually understand differ- 
ent strategical scenarios. Generally, game 
theory investigates conflict situations, the 
interaction between the agents and their 
decisions. 

A game in the sense of game theory 
is given by a (mostly finite) number of 
players, who interact according to given 
rules. Those players might be individu- 
als, groups, companies, associations and 
so on. Their interactions will have an im- 
pact on each of the players and on the 
whole group of players, i.e. they are in- 
terdependent. 

To be more precise: A game is de- 
scribed by a set of players and their possi- 
bilities to play the game according to the 
rules, i.e. their set of strategies. 


This description leads to a widespred 
definition of game theory: 


The subject of game theory 
are situations, where the re- 
sult for a player does not only 
depend on his own decisions, 
but also on the behaviour of 
the other players. 


Game theory has its historical origin 
in 1928. By analysing parlour games, John 
von Neumann realised very quickly the 
practicability of his approaches for the 
analysis of economic problems. 

In his book Theory of Games and Eco- 
nomic Behavior, which he wrote together 
with Oskar Morgenstern in 1944, he al- 
ready applied his mathematical theory to 
economic applications. 

The publication of this book is gen- 
erally seen as the initial point of modern 
game theory. 


1.2 Game Theory — Where is it 
applied? 


As we have seen in the previous section, 
game theory is a branch of mathemat- 
ics. Mathematics provide a common lan- 
guage to describe these games. We have 
also seen that game theory was already 
applied to economics by von Neumann. 
When there is competition for a resource 
to be analysed, game theory can be used 
either to explain existing behaviour or to 
improve strategies. 

The first is especially applied by sci- 
ences which analyse long-term situations, 
like biology or sociology. In animality, for 
example, one can find situations, where 
cooperation has developed for the sake of 
mutual benefits. 

The latter is a crucial tool in sciences 
like economics. Companies use game the- 
ory to improve their strategical situation 
in the market. 
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Despite the deep insights he gained from 
game theory’s applications to economics, 
von Neumann was mostly interested in 
applying his methods to politics and war- 
fare, perhaps descending from his favorite 
childhood game, Kriegspiel, a chess-like 
military simulation. He used his meth- 
ods to model the Cold War interaction be- 
tween the U.S. and the USSR, picturing 
them as two players in a zero-sum game. 

He sketched out a mathematical model 
of the conflict from which he deduced that 
the Allies would win, applying some of 
the methods of game theory to his pre- 
dictions. 

There are many more applications in 
the sciences, which have already been men- 
tioned, and in many more sciences like so- 
ciology, philosophy, psychology and cul- 
tural anthropology. It is not possible to 
list them all in this paper, more informa- 
tion can be obtained in the references at 
the end of this paper. 


2 Definitions 


I now introduce some of the basic defini- 


tions of game theory. I use a non-mathematical 


description as far as possible, since math- 
ematics is not really required to under- 
stand the basic concepts of game theory. 

However, a mathematical derivation is 
given in appendix A.1 and A.2. 


2.1 Normal Form Games 


A game in normal form consists of: 


1. A finite number of players. 


2. Astrategy set assigned to each player. 
(e.g. in the Prisoner’s Dilemma each 
player has the possibility to cooper- 
ate (C) and to defect (D). Thus his 
strategy set consists of the elements 
C and D.) 


3. A payoff function, which assigns a 
certain payoff to each player depend- 
ing on his strategy and the strat- 
egy of the other players (e.g. in the 
Prisoner’s Dilemma the time each of 
the players has to spend in prison). 


The payoff function assigns each player 
a certain payoff depending on his strat- 
egy and the strategy of the other play- 
ers. If the number of players is limited 
to two and if their sets of strategies con- 
sist of only a few elements, the outcome 
of the payoff function can be represented 
in a matrix, the so-called payoff matriz, 
which shows the two players, their strate- 
gies and their payofts. 


Example: 


PlayerllPlayer2 | L R 
U 1, 
D 1 


wr 
w| e 


In this example, player 1 (vertical) has 
two different strategies: Up (U) and Down 
(D). Player 2 (horizontal) also has two 
different strategies, namely Left (L) and 
Right (R). 

The elements of the matrix are the 
outcomes for the two players for playing 
certain strategies, i.e. supposing, player 1 
chooses strategy U and player 2 chooses 
strategy R, the outcome is (2, 4), i.e. the 
payoff for player 1 is 2 and for player 2 is 
4. 


2.2 Extensive Form Games 


Contrary to the normal form game, the 
rules of an extensive form game are de- 
scribed such that the agents of the game 
execute their moves consecutively. 

This game is represented by a game 
tree, where each node represents every 
possible stage of the game as it is played. 
There is a unique node called the initial 
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node that represents the start of the game. 
Any node that has only one edge con- 
nected to it is a terminal node and rep- 
resents the end of the game (and also a 
strategy profile). Every non-terminal node 
belongs to a player in the sense that it 
represents a stage in the game in which it 
is that player’s move. Every edge repre- 
sents a possible action that can be taken 
by a player. Every terminal node has 
a payoff for every player associated with 
it. These are the payoffs for every player 
if the combination of actions required to 


reach that terminal node are actually played. 


Example: 


Figure 1: A game in extensive form 


In figure 1 the payoff for player 1 will be 
2 and for player 2 will be 1, provided that 
player 1 plays strategy U and player 2 
plays strategy D’. 


2.3 Nash Equilibrium 


In game theory, the Nash equilibrium (named 


after John Nash, who first described it) 
is a kind of solution concepts of a game 
involving two or more players, where no 
player has anything to gain by changing 
only his own strategy. 

If each player has chosen a strategy 
and no player can benefit by changing 
his strategy while the other players keep 
theirs unchanged, then the current set of 


strategy choices and the corresponding pay- 
offs constitute a Nash equilibrium. 

John Nash showed in 1950, that every 
game with a finite number of players and 
finite number of strategies has at least one 
mixed strategy Nash equilibrium. 


2.3.1 Best Response 


The best response is the strategy (or strate- 
gies) which produces the most favorable 
immediate outcome for the current player, 
taking other players’ strategies as given. 

With this definition , we can now de- 
termine the Nash equilibrium in a normal 
form game very easily by using the payoff 
matrix. 

The formal proof that this procedure 
leads to the desired result is given in ap- 
pendix A.2. 


2.3.2 Localizing a Nash Equilibrium 
in a Payoff-matrix 


Let us use the payoff matrix of the Pris- 
oner’s Dilemma, which will be introduced 
in 3.1, to determine the Nash equilibrium: 


Playerl\Player2 | C D 
C 3,3 0,5 
D 50 11 


The procedure is the following: First we 
consider the options for player 1 by a given 
strategy of player 2, i.e. we look for the 
best answer to a given strategy of player 


If player 2 plays C, the payoff for player 
1 for coosing C will be 3, for choosing D it 
will be 5, so we highlight his best answer, 
D: 


C 
D 


ol% 
=| © 
=| o 


ajo 


Now we repeat this procedure for the case, 
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that player 2 plays D (the best answer 
in this case is D, since C gains payoff 0 
whereas D gains payoff 1), then we do the 
same for player 2 by a given strategy of 
player 1 and we will get: 


C 
D 


olw 
u| © 
R| Or 


ajo 


The Nash equilibrium is then determined 
by the matrix element, in which both play- 
ers marked their best answers. 

Thus, the strategies that constitute 
a Nash equilibrium is defection by both 
players, because if any player changed his 
strategy to C whereas the other one stays 
with D, he would get a less payoff. 


2.4 Mixed Strategies 


Consider the following payoff matrix, which 


corresponds to the game Matchin Pen- 
nies: 


Player1\Player2 | Head | Tail | 
Head 1,-1|-1,1| 
Tail -1,1[1,-1| 


The best responses are already marked, 
and it is obvious, that there is no matrix 
cell, in which both players marked their 
best response. 

What do game theorists make of a 
game without a Nash equilibrium? The 
answer is that there are more ways to play 
the game than are represented in the ma- 
trix. Instead of simply choosing Head or 
Tail, a player can just flip the coin to de- 
cide what to do. This is an example of 
a mixed strategy, which simply means a 


particular way of choosing randomly among 


the different strategies. 

The mixed strategy equilibrium of the 
matching pennies game is well known: each 
player should randomize 50-50 between 
the two alternatives. 


Mixed strategy equilibrium points out 
an aspect of Nash equilibrium that is of- 
ten confusing for beginners. Nash equi- 
librium does not require a positive reason 
for playing the equilibrium strategy. In 
matching pennies, the two players are in- 
different: they have no positive reason to 
randomize 50-50 rather than doing some- 
thing else. However, it is only an equilib- 
rium if they both happen to randomize 
50-50. The central thing to keep in mind 
is that Nash equilibrium does not attempt 
to explain why players play the way they 
do. It merely proposes a way of playing 
so that no player would have an incentive 
to play differently. 

If, for example, player 1 chooses to 
play Head with a probability of 80 % and 
play Tail with a probability of 20 % , then 
player 2 will eventually anticipate the op- 
ponent's strategy. Hence he will play Tail 
everytime. This will lead to a positive 
payoff of 0.8 - 1 + 0.2 - (—1) = 0.6 per 
game for player 2, respectively a payoff 
of —0.6 for player 1. The best payoff one 
can do in such a fair zero-sum game is 
0, and this will be achieved by playing 
$ Head + 3 Tail. 

Mathematically spoken, a mixed strat- 
egy is just a linear combination of the 
pure strategies. 


3 Games 


Now I want to present some of the most 
studied games of game theory. 


3.1 Prisoner’s Dilemma (PD) 


We have already used the payoff matrix 
of the Prisoner’s Dilemma in 2.3.2 to lo- 
calize the Nash equilibrium of this game, 
and now I want to tell the story behind 
this famous dilemma: 


Two suspects are arrested by the police. 
The police have insufficient evidence for 
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a conviction, and, having separated both 
prisoners, an officer visits each of them 
to offer the same deal: if one testifies for 
the prosecution against the other and the 
other remains silent, the betrayer goes free 
and the silent accomplice receives the full 
10-year sentence. If both stay silent, the 
police can sentence both prisoners to only 
six months in jail for a minor charge. If 
each betrays the other, each will receive 
a two-year sentence. Each prisoner must 
make the choice of whether to betray the 
other or to remain silent. However, nei- 
ther prisoner knows for sure what choice 
the other prisoner will make. So the ques- 
tion this dilemma poses is: How will the 
prisoners act? 


We will use the following abbreviations: 
To testify means to betray the other sus- 
pect and thus to defect (D), to remain 
silent means to cooperate (C) with the 
other suspect. And for the sake of clarity, 
we want to use positive numbers in the 
payoff matrix. 


C D 
C | R=3, R 
T=5, S 


> 


3 
0 


e Risa Reward for mutual coopera- 
tion. Therefore, if both players co- 
operate then both receive a reward 
of 3 points. 


If one player defects and the other 
cooperates then one player receives 
the Temptation to defect payoff (5 
in this case) and the other player 
(the cooperator) receives the Sucker 
payoff (zero in this case). 


If both players defect then they both 
receive the Punishment for mutual 
defection payoff (1 in this case). 


As we have already seen, the logical move 


for both players is defection (D). The dilemma 


lies herein, that the best result for player 
l and player 2 as a group (R = 3 for both) 
can't be achieved. 


In defining a PD, certain conditions have 
to hold. The values we used above, to 
demonstrate the game, are not the only 
values that could have been used, but they 
do adhere to the conditions listed below. 

Firstly, the order of the payoffs is im- 
portant. The best a player can do is T 
(temptation to defect). The worst a player 
can do is to get the sucker payoff, S. If 
the two players cooperate then the reward 
for that mutual cooperation, R, should 
be better than the punishment for mu- 
tual defection, P. Therefore, the following 
must hold: 


T>R>P>S. 


In repeated interactions, another condi- 
tion it is additionally required: 

Players should not be allowed to get 
out of the dilemma by taking it in turns to 
exploit each other. Or, to be a little more 
pedantic, the players should not play the 
game so that they end up with half the 
time being exploited and the other half 
of the time exploiting their opponent. In 
other words, an even chance of being ex- 
ploited or doing the exploiting is not as 
good an outcome as both players mutu- 
ally cooperating. Therefore, the reward 
for mutual cooperation should be greater 
than the average of the payoff for the temp- 
tation and the sucker. That is, the follow- 
ing must hold: 


R> (S +T)/2 


3.1.1 Other Interesting Two-person 
Games 


Depending on the order of R, T, S, and 
P, we can have different games. Most are 
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trivial, but two games stand out: 


e Chicken (T > R>S > P) 


C D 
2, R=2 
3, S=1 


CIR 
DIT 


S=1, T=3 
P=0, P=0 


Example: Two drivers with something to 
prove drive at each other on a narrow 
road. The first to swerve loses faces among 
his peers (the chicken). If neither swerves, 
however, the obvious worst case will oc- 
cur. 


e Stag Hunt (R>T>P>S) 


C D 
C|R=3,R 0, T=2 
D | T=2,8 1, P=1 


3/5 
0|P 


> 


Example: Two hunters can either jointly 
hunt a stag or individually hunt a rabbit. 
Hunting stags is quite challenging and re- 
quires mutual cooperation. Both need to 
stay in position and not be tempted by 
a running rabbit. Hunting stags is most 
beneficial for society but requires a lot of 
trust among its members. The dilemma 
exists because you are afraid of the oth- 
ers’ defection. Thus, it is also called trust 
dilemma. 


3.2 The Ultimatum Game 


Imagine you and a friend of yours are 
walking down the street, when suddenly 
a stranger stops you and wants to play a 
game with you: 

He offers you 100$ and you have to 
agree on how to split this money. You, 
as the proposer, make an offer to your 
friend, the responder. If he accepts your 
offer, the deal goes ahead. If your friend 
rejects, neither player gets anything. The 
stranger will take back his money and the 
game is over. 


Obviously, rational responders should 
accept even the smallest positive offer, since 
the alternative is getting nothing. Pro- 
posers, therefore, should be able to claim 
almost the entire sum. In a large num- 
ber of human studies, however, conducted 
with different incentives in different coun- 
tries, the majority of proposers offer 40 to 
50% of the total sum, and about half of 
all responders reject offers below 30% 


33 Public Good Game 


A group of 4 people are given $200 each 
to participate in a group investment project. 
They are told that they could keep any 
money they do not invest. The rules of 
the game are that every $1 invested will 
yield $2, but that these proceeds would 
be distributed to all group members. If 
every one invested, each would get $400. 
However, if only one person invested, that 
“sucker” would take home a mere $100. 
Thus, the assumed Nash equilibrium could 
be the combination of strategies, where 
no one invests any money. And we can 
show that this is indeed the Nash equilib- 
rium. 

We will not display this game in a pay- 
off matrix, since each player has a too big 
set of strategies (the strategy sn is given 
by the amount of money that player n 
wants to contribute, e.g. sı = 10 means, 
that player 1 invests 10$). Nevertheless 
this is a game in normal form and there- 
fore it has a payoff function for each player. 
The payoff function for, let’s say, player 1 
is given by 


p 2 (sit s2 +83 + 84) if 
4 
2. 
= (sa = + $4) 0,5 eS) 


But this means, that every investment 
sı of player 1 will diminish his payoff. 
Therefore, a rational player will choose 
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the strategy sy = 0, i.e. he will invest no 
money at all. But if everyone plays this 
strategy, no one can benefit by changing 
his strategy while the other players keep 
their strategy unchanged. And this is just 
the definition of the Nash equilibrium. 

The dilemma lies exactly herein, since 
the greatest benefit for the whole group 
arises, if everyone contributed all of his 
money. 


3.4 Rock, Paper, Scissors 


The players simultaneously change their 
fists into any of three ” objects”: 


e Rock: a clenched fist. 


e Paper : all fingers extended, palm 
facing downwards, upwards, or side- 
ways. 


Scissors: forefinger and middle fin- 
ger extended and separated into a 
”V” shape. 


The objective is to defeat the oppo- 
nent by selecting a weapon which defeats 
their choice under the following rules: 


1. Rock crushes Scissors (rock wins) 
2. Scissors cut Paper (scissors win) 


3. Paper covers Rock and roughness is 
covered (paper wins) 


If players choose the same weapon, the 
game is a tie and is played again. 

This is a classic non-transitive system 
which involves a community of three com- 
peting species satisfying a relationship. 

Such relationships have been demon- 
strated in several natural systems. 


4 Evolutionary Game The- 
ory 


Since game theory was established as a 
discipline for its own, it has been very 
successful. 

However, there have been situations, 
which could not be properly described by 
the language of game theory. 

Many people have attempted to use 
traditional game theory to analyze eco- 
nomic or political problems, which typi- 
cally involve a large population of agents 
interacting. However, traditional game 
theory is a “static” theory, which reduces 
its usefulness in analyzing these very sorts 
of situations. EGT improves upon tra- 
ditional game theory by providing a dy- 
namics describing how the population will 
change over time. Therefore a new math- 


ematical extension has been developed (mainly 


by John Maynard Smith in his book Evo- 
lution and the Theory of Games, 1982), 
which is called evolutionary game theory 
(EGT) 

I will show in the next section, in what 
kinds of situations EGT might be appli- 
cable and what are the most significant 
differences to game theory. 


4.1 Why EGT? 


Evolutionary game theory (EGT) stud- 
les equilibria of games played by a pop- 
ulation of players, where the fitness of 
the players derives from the success each 
player has in playing the game. It pro- 
vides tools for describing situations where 
a number of agents interact and where 
agents might change the strategy they fol- 
low at the end of any particular interac- 
tion. 

So, the questions of EGT are: Which 
populations are stable? When do the in- 
dividuals adopt other strategies? Is it 
possible for mutants to invade a given pop- 
ulation? 
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Another very prominent application is 
the quest for the origins and evolution 
of cooperation. The effects of population 
structures on the performance of behav- 
ioral strategies became apparent only in 
recent years and marks the advent of an 
intriguing link between apparently unre- 
lated disciplines. EGT in structured pop- 
ulations reveals critical phase transitions 
that fall into the universality class of di- 
rected percolation on square lattices. 

Together with EGT as an extension 
of game theory, new concepts were devel- 
oped to investigate and to describe these 
very problems. I will now introduce two 
of them, which are crucial to describe EGT, 
evolutionary stable strategies and the repli- 
cator dynamics. The first one is applied 
tu study the stability of populations, the 


latter one investigates the adoption of strate- 


gies. 


4.2 Evolutionary Stable Strate- 
gies 


An evolutionary stable strategy (ESS) is 
a strategy which if adopted by a popula- 
tion cannot be invaded by any competing 
alternative strategy. The concept is an 
equilibrium refinement to the Nash equi- 
librium. 

The definition of an ESS was intro- 
duced by John Maynard Smith and George 
R. Price in 1973 based on W.D. Hamil- 
ton’s (1967) concept of an unbeatable strat- 
egy in sex ratios. The idea can be traced 
back to Ronald Fisher (1930) and Charles 
Darwin (1859). 


4.2.1 ESS and Nash Equilibrium 


A Nash equilibrium is a strategy in a game 
such that if all players adopt it, no player 
will benefit by switching to play any al- 
ternative strategy. 

If a player, choosing strategy u in a 
population where all other players play 


strategy a, receives a payoff of E(p, 0), 
then strategy o is a Nash equilibrium if 
E(o,c) > El(p,0), ie. o does just as 
good or better playing against o than any 
mutant with strategy u does playing against 
O. 

This equilibrium definition allows for 
the possibility that strategy y is a neutral 
alternative to o (it scores equally, but not 
better). A Nash equilibrium is presumed 
to be stable even if u scores equally, on 
the assumption that players do not play 
TA 

Maynard Smith and Price specify (May- 
nard Smith & Price, 1973; Maynard Smith 
1982) two conditions for a strategy o to 
be an ESS. Either 


1. E(o,0) > E(p, 0), or 


2. Elo,0) = Elp,0) and 
E(o, u) > E(u, p) 


must be true for all o Æ y. 

In other words, what this means is 
that a strategy o is an ESS if one of two 
conditions holds: 


1. o does better playing against o than 
any mutant does playing against O, 
or 


2. some mutant does just as well play- 
ing against O as ø, but o does better 
playing against the mutant than the 
mutant does. 


A derivation of ESS is given in ap- 
pendix A.3. 


4.2.2 The Hawk-Dove Game 


As an example for ESS, we consider the 
Hawk-Dove Game. In this game, two in- 
dividuals compete for a resource of a fixed 
value V (The value V of the resource cor- 
responds to an increase in the Darwinian 
fitness of the individual who obtains the 
resource). Each individual follows exactly 
one of two strategies described below: 
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e Hawk: Initiate aggressive behaviour, 
not stopping until injured or until 
one’s opponent backs down. 


e Dove: Retreat immediately if one’s 


opponent initiates aggressive behaviour. 


If we assume that 


1. whenever two individuals both ini- 
tiate aggressive behaviour, conflict 
eventually results and the two indi- 
viduals are equally likely to be in- 
jured, 


2. the cost of the conflict reduces indi- 
vidual fitness by some constant value 
C, 


3. when a hawk meets a dove, the dove 
immediately retreats and the hawk 
obtains the resource, and 


4. when two doves meet the resource 
is shared equally between them, 


the fitness payoffs for the Hawk-Dove game 
can be summarized according to the fol- 
lowing matrix: 


Hawk Dove 
Hawk | $(V — C), 4(V —C) V,0 
Dove 0, V V/2,V/2 


One can readily confirm that, for the Hawk- 
Dove game, the strategy Dove is not evo- 
lutionarily stable because a pure popula- 
tion of Doves can be invaded by a Hawk 
mutant. If the value V of the resource 
is greater than the cost C of injury (so 
that it is worth risking injury in order 
to obtain the resource), then the strat- 
egy Hawk is evolutionarily stable. In the 
case where the value of the resource is less 
than the cost of injury, there is no ESS 
if individuals are restricted to following 
pure strategies, although there is an ESS 
if players may use mixed strategies. 
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4.2.3 ESS of the Hawk-Dove Game 


Clearly, Dove is no stable strategy, since 


Y = E(D,D) < E(H,D) = V, a popu- 
lation of doves can be invaded by hawks. 


Because of E(H, H) = $(V—C) and E(D, H) = 


0, H is an ESS if V > C. But what if 
V < C? Neither H nor D is an ESS. 

But we could ask: What would hap- 
pen to a population of individuals which 
are able to play mixed strategies? Maybe 
there exists a mixed strategy which is evo- 
lutianary stable. 

Consider a population consisting of a 
species, which is able to play a mixed 
strategy I, i.e. sometimes Hawk and some- 
times Dove with probabilities p and 1—p 
respectively. 

For a mixed ESS I to exist the follow- 
ing must hold: 


E(D, I) = E(H,1) = E(I,1) 


Suppose that there exists an ESS in 
which H and D, which are played with 
positive probability, have different pay- 
offs. Then it is worthwile for the player 
to increase the weight given to the strat- 
egy with the higher payoff since this will 
increase expected utility. 

But this means that the original mixed 
strategy was not a best response and hence 
not part of an ESS, which is a contradic- 
tion. Therefore, it must be that in an 
ESS all strategies with positive probabil- 
ity yield the same payoff. 

Thus: 


E(H,1I) = E(D,1) 


= pE(H,H)+( 
pE(D,H)+(1-p 


s 2 =H 40 =pv= 
(l—p)} 


V 


4 EVOLUTIONARY GAME THEORY 


Thus a mixed strategy with a proba- 
bility V/C of playing Hawk and a proba- 
bility 1 — V/C of playing Dove is evolu- 
tionary stable, i.e. that it can not be in- 
vaded by players playing one of the pure 
strategies Hawk or Dove. 


4.3 The Replicator Dynamics 


As mentioned before, the main difference 
of EGT to game theory is the investiga- 
tion of dynamic processes. In EGT, we 
are interested in the dynamics of a pop- 
ulation, i.e. how the population evolves 
over time. 


Let us consider now a population consist- 
ing of n types, and let x;(t) be the fre- 
quency of type i. Then the state of the 
population is given by the vector x(t) = 
al. Enlt). 

We want now to postulate a law of 
motion for x(t). If individuals meet ran- 
domly and then engage in a symmetric 
game with payoff matrix A, then (Ax); is 
the expected payoff for an individual of 
type i and x! Ax is the average payoff in 
the population state x. 

The evolution of x over time is de- 
scribed by the replicator equation: 


a, = x;[(Ax); — xT Ax] (1) 


The replicator equation describes a se- 
lection process: more succesful strategies 
spread in the population. 

A derivation of the replicator equation 
is given in appendix A.4 


4.4 ESS and Replicator Dynam- 
ics 


We have seen, that D is no ESS at all and 
for V > C, H is an ESS. We have also 
seen, that for V < C the ESS is a mixed 
ESS with a probability of V/C for playing 
H and with a probability of 1 — V/C for 
playing D. 
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By setting č; = 0, we obtain the evo- 
lutionary stable states of a population. 

A population is said to be in an evo- 
lutionarily stable state if its genetic com- 
position is restored by selection after a 
disturbance, provided the disturbance is 
not too large. 

If this equation is applied to the Hawk- 
Dove game, the result will be the follow- 
ing: 

For V > C, the only evolutionary sta- 
ble state is a population consisting of hawks. 
For V < C, a mixed population with a 
fraction V/C of hawks and a fraction 1 — 
V/C of doves is evolutionary stable. 

This result will be derived in appendix 
A.D. 

At this point, one may see little differ- 
ence between the two concepts of evolu- 
tionary game theory. We have confirmed 
that, for the Hawk-Dove game and for 
V > C, the strategy Hawk is the only 
ESS. Since this state is also the only sta- 
ble equilibrium under the replicator dy- 
namics, the two notions fit together quite 
neatly: the only stable equilibrium under 
the replicator dynamics occurs when ev- 
eryone in the population follows the only 
ESS. In general, though, the relationship 
between ESSs and stable states of the repli- 
cator dynamics is more complex than this 
example suggests. 

If only two pure strategies exist, then 
given a (possibly mixed) evolutionarily sta- 
ble strategy, the corresponding state of 
the population is a stable state under the 
replicator dynamics. (If the evolutionar- 
ily stable strategy is a mixed strategy S, 
the corresponding state of the population 
is the state in which the proportion of 
the population following the first strategy 
equals the probability assigned to the first 
strategy by S, and the remainder follow 
the second strategy.) 

However, this can fail to be true if 
more than two pure strategies exist. 


5 APPLICATIONS 


The connection between ESSs and stable 
states under an evolutionary dynamical 
model is weakened further if we do not 
model the dynamics by the replicator dy- 
namics. 

In 5.1 we use a local interaction model 
in which each individual plays the Pris- 
oner’s Dilemma with his or her neighbors. 
Nowak and May, using a spatial model in 
which local interactions occur between in- 
dividuals occupying neighboring nodes on 
a square lattice, showed that stable pop- 
ulation states for the Prisoner’s Dilemma 
depend upon the specific form of the pay- 
off matrix. 


5 Applications 


5.1 Evolution of cooperation 


As mentioned before, the evolution of co- 
operation is a fundamental problem in bi- 
ology because unselfish, altruistic actions 
apparently contradict Darwinian selection. 

Nevertheless, cooperation is abundant 
in nature ranging from microbial interac- 
tions to human behavior. In particular, 
cooperation has given rise to major tran- 
sitions in the history of life. Game theory 
together with its extensions to an evolu- 
tionary context has become an invaluable 
tool to address the evolution of coopera- 
tion. The most prominent mechanisms of 
cooperation are direct and indirect reci- 
procity and spatial structure. 

The mechanisms of reciprocity can be 
investigated very well with the Ultima- 
tum game and also with the Public Good 
game. 

But the prime example to investigate 
spatially structured populations is the Pris- 
oner’s Dilemma. 

Investigations of spatially extended sys- 
tems have a long tradition in condensed 
matter physics. Among the most impor- 
tant features of spatially extended sys- 
tems are the emergence of phase transi- 
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tions. Their analysis can be traced back 
to the Ising model. The application of 
methods developed in statistical mechan- 
ics to interactions in spatially structured 
populations has turned out to be very fruit- 
ful. Interesting parallels between non equi- 
librium phase transitions and spatial evo- 
lutionary game theory have added another 
dimension to the concept of universality 
classes. 

We have already seen, that the Nash 
equilibrium of PD is to defect. But to 
overcome this dilemma, we consider spa- 
tially structured populations where indi- 
viduals interact and compete only within 
a limited neighborhood. Such limited lo- 
cal interactions enable cooperators to form 
clusters and thus individuals along the 
boundary can outweigh their losses against 
defectors by gains from interactions within 
the cluster. Results for different popula- 
tion structures in the PD are discussed 
and related to condensed matter physics. 


This problem has been investigated by 
Martin Nowak (Nature, 359, pp. 826- 
829, 1992) 

I programmed this scenario based on 
the investigations of Nowak. The pro- 
gram is written in NetLogo, the program 
code is given in appendix B. 


5.2 Biodiversity 


One of the central aims of ecology is to 
identify mechanisms that maintain bio- 
diversity. Numerous theoretical models 
have shown that competing species can 
coexist if ecological processes such as dis- 
persal, movement, and interaction occur 
over small spatial scales. In particular, 
this may be the case for nontransitive com- 
munities, that is, those without strict com- 


petitive hierarchies. The classic non-transitive 


system involves a community of three com- 
peting species satisfying a relationship sim- 


ilar to the children's game rock-paper-scissors, 


5 APPLICATIONS 


where rock crushes scissors, scissors cuts 
paper, and paper covers rock. Such rela- 
tionships have been demonstrated in sev- 
eral natural systems. Some models pre- 
dict that local interaction and dispersal 
are sufficient to ensure coexistence of all 
three species in such a community, whereas 
diversity is lost when ecological processes 
occur over larger scales. Kerr et al., tested 
these predictions empirically using a non- 


transitive model community containing three 


populations of Escherichia coli. They found 
that diversity is rapidly lost in our experi- 
mental community when dispersal and in- 
teraction occur over relatively large spa- 
tial scales, whereas all populations coexist 
when ecological processes are localized. 

There exist three strains of Escherichia 
coli bacteria: 


e Type A releases toxic colicin and 
produces, for its own protection, an 
immunity protein. 


e Type B produces the immunity pro- 
tein only. 


e Type C produces neither toxin nor 
immunity. 


The production of the toxic colicin and 
the immunity protein causes some higher 
costs. Thus the strain which produces 
the toxin colicin is superior to the strain 
which has no immunity protein (A beats 
C). 

The one with no immunity protein is 
superior to the strain with immunity pro- 
tein, since it has lower costs to reproduce 
(C beats B) 

The same holds for the strain with im- 
munity protein but no production of the 
colicin compared to the strain which pro- 
duces colicin (B beats A). 


In figure 2, one can see, that on a 
static plate, which is an environment in 
which dispersal and interaction are pri- 
marily local, the three strain coexist. 
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a Static Plate 
12 


Log (abundance) 


Figure 2: Escherichia coli on a static plate 


green: resistant strain 
red: colicin producing strain 
blue: sensitive strain 


Figure 3: Escherichia coli in a flask 


In figure 3, where the strains are held 
in a flask, a well-mixed environment in 
which dispersal and interaction are not 
exclusively local, only the strain, which 
produces the immunity protein only will 
survive. 


A MATHEMATICAL DERIVATION 


A Mathematical Derivation 


A.1 Normal form games 
A game in normal form consists of: 


1. A (finite) number of players M = 
{al,..., an} 


2. A strategy set S; assigned to each 
player i € M. 
The combination of all sets of strate- 
gies S = ] [iem Si is called strategy 
space. 


3. A utility/payoff function u; : S > 
R, assigned to each player i € M. 
=>VseES:ui(s) ER 


A.2 Nash equilibrium and best 
answer 


Notations: 


1. s€ S= [liem Si; 
lei SM); si € Si 


Dn A Si e SM) 
(S85) = 8% (844,57) S 


3. Si = ILjem, E 
Six Sa =s: (Si, S—i) E Si x S_; 


Definition: 
A combination of strategies s* € S is called 


a Nash equilibrium iff: 


Vie M Vs; € S;: 
ass s*;) 0 (88%) 


Definition: 


A strategy $; € S; is called best answer 
to a combination of strategies s_; € S_; 
iff: 


Vs; E S; : uil $i, si) > ua (Sis si) 
> us (Si, s-i) = max{u;(s;, S-i); si € Si} 
It is now easy to see, that, if every 
player chooses his best answer, this will 
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constitute a Nash equilibrium, since an- 
other strategy will not lead to an increase 
of the payoff, since the best answer al- 
ready leads to the maximum payoff. 


A.3 Evolutionary stable strate- 
gies (ESS) 


In order for a strategy to be evolutionar- 
ily stable, it must have the property that 
if almost every member of the population 
follows it, no mutant (that is, an individ- 
ual who adopts a novel strategy) can suc- 
cessfully invade. This idea can be given 
a precise characterization as follows: Let 
E(s1, 82) denote the payoff (measured in 
Darwinian fitness) for an individual fol- 
lowing strategy sı against an opponent 
following strategy s2, and let W(s) denote 
the total fitness of an individual follow- 
ing strategy s; furthermore, suppose that 
each individual in the population has an 
initial fitness of Wo. 

We consider now a population consist- 
ing mainly of individulas following strat- 
egy o which shall be an ESS with a small 
frequency p of some mutants playing p. 
Then if each indivudal engages in one con- 
test 


Since o is evolutionarily stable, the 
fitness of an individual following o must 
be greater than the fitness of an individ- 
ual following (otherwise the mutant fol- 
lowing u would be able to invade), and so 
W(o) > W(u). Now, as p is very close to 
0, this requires either that 


E(o,0) > Elp, 0) 


A MATHEMATICAL DERIVATION 


A.4 Replicator equation 


The fitness of a population is given by 
(Ax); and the total fitness of the entire 
population is given by xT Ax. Thus, the 
relative fitness of a population is given by 


(Ax); 
xT Ax 


Let us assume that the proportion of the 
population following the strategies in the 
next generation is related to the propor- 
tion of the population following the strate- 
gies current generation according to the 
rule: 


for xT Ax 4 0. Thus 


ntan -ale = ay EN 2 


This yields the differential equation for 
At — 0: 


(Ax); — x? Ax 
q; = HA Ax] (2) 


fori=1,...,n with č; denoting the deriva- 
tive of x; after time. 
The simplified equation 


a; = x;[(Ax); — xT Ax] (3) 


has the same trajectories than (2), since 
every solution x(t) of (2) delivers accord- 
ing to the time transformation 


t(s) = ie x(t)" Ax(t)dt 


a solution y(s) := x(t(s)) of (3). 
Equation (3) is called the replicator 
equation. 


A.5 Evolutionary stable state of 
the Hawk-Dove game 


We want to show, that the replicator dy- 
namics and ESS yield to the same result 
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for the Hawk-Dove game. 


The replicator equation is given by 
a; = x;[(Ax); — xT Ax)] 


Let us denote the population of hawks x, 
with p, thus the population of doves x 
will be denoted with 1—p. The first term 
Ax gives 


ion o) V(I- p) ) 
3 (l —p) 
Since Hawk is denoted with x1, we will 


use the first component of the vector Ax. 
The second term x! Ax delivers 


5-0) +pV(1-p) + 5(1- p)? 
Thus: 
p = piKV—-C)+V0-p)-5(V-C) 
pV(1—p) - $(1-p)?| 
= p[5p? — 3(V + C)p+ 5] 
= plp?- p+ $] 


In order to be a population evolutionary 
stable we set the changes of the popula- 
tion per time to zero, so that there is no 
change in time. This gives: 


p=0 
> plp?- “t2p+ ¥]=0 
This is certainly true for p = 0. This is 
the trivial solution. Two other solutions 


can be obtained by evaluating the term 
in the brackets: 


2 VtC V 


+= =0 
E 
This gives 
_ V4C y /V242VC+C2 YV 
Pp = 3 + 402 C 
_ V4C y, /V2-2VC+C2 
a IG = 4C2 
_ V+C 1 V-C 


A MATHEMATICAL DERIVATION 


Thus: 

zj A 

Pı > P2 C 
pı = 1 is another trivial solution, thus 

the only relevant result is pa = £. 
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B POGRAM CODE 


B Pogram Code 


B.1 


globals 


Spatial PD 


[movie_on?] 


patches-own [num_D z z_prev score d score_h neighbor_h] 


to setup 


ca 
551/3 


will be defectors (z=1, red); 2/3 cooperators (z=0; blue) 


ask patches [ifelse ((random 3) < 1) [set pcolor red set z 1][set pcolor blue set z 0]] 
3 ;d=delta*(2.0*(random(1.0)-1.0)) 

ask patches [set d delta * (2.0 * (random-float 1.0) - 1.0)] 

set movie_on? false 


end 


to single-D 


ca 


ask patches [set pcolor blue set z 0] 
ask patches [set d delta * (2.0 * (random-float 1.0) - 1.0)] 
ask patch 0 O [set pcolor red set z 1] 


set movie_on? false 
end 
to go 
play-game 
update 
if (movie_on?) [movie-grab-view] 
end 
to play-game 
ask patches [set z_prev z] 
ask patches 
C 
; num_D = number of neighbors that are defectors 
set num_D nsum z 
ifelse z = 1 
[ 
; if patch is defector: score is T times number of cooperators (8-num_D) 
set score (T * (8 - num_D)) + (P * (num_D + 1)) 
] 
[ 
; if patch is cooperator: score is 1=R times number of cooperators (+1=itself) 
set score (R * (9 - num_D)) + (S * num_D) 
] 
] 
end 
to update 


; find the neighbor with the highest payoff 
ask patches 


[ 


; neighbor_h = the neighbor with the highest score 
set neighbor_h max-one-of neighbors [score] 

3 score_h = highest score 

set score_h score-of neighbor_h 


ifelse (score_h >= score + d) 
[ 
; if highest score > own score (+d) 
ifelse z_prev = 1 
[ 
; if this patch is defector: 
; if neighbor with highest score is cooperator: set patch to cooperator (z=0, green) 
; else: stay defector 
ifelse (z_prev-of neighbor_h) = 0 [set pcolor green set z 0][set pcolor red] 
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B POGRAM CODE 18 


] 
[ 
; if this patch is cooperator: 
; if neighbor with highest score is defector: set patch to defector (z=1, yellow) 
; else: stay cooperator 
ifelse (z_prev-of neighbor_h) = 1 [set pcolor yellow set z 1][set pcolor blue] 
] 
set d delta * (2.0 * (random-float 1.0) - 1.0) 
] 
[ 
; if own score is the highest: 
; if cooperator: set color blue, otherwise red 
ifelse z = 0 [set pcolor blue] [set pcolor red] 
] 
] 
end 
to perturb 


if mouse-down? 


[ 


ask patch-at mouse-xcor mouse-ycor [set z 1 - z ifelse z = 0 [set pcolor blue] [set pcolor red]] 
¡need to wait for a while; otherwise the procedure is run a few times after a mouse click 
wait 0.5 

] 


end 


to movie_start 
movie-start "out.mov" 
set movie_on? true 
end 


to movie_stop 
movie-close 
set movie_on? false 
end 


B POGRAM CODE 


B.2 Hawk-Dove game 


breed [hawks hawk] 

breed [doves dove] 

globals [deltaD deltaH p total counter decrement reproduce_limit] 
turtles-own [energy] 


to setup 
ca 
set-default-shape hawks "hawk" 
set-default-shape doves "butterfly" 


createH n_hawks 
createD n_doves 


ask turtles [set energy random-float 10.0] 


set reproduce_limit 11.0 
set decrement 0.05 
end 


to go 
ask turtles 
[ 
move 
fight 
reproduce 


] 


while [count turtles > 600] 
[ask one-of turtles[die]] 


do-plot 
end 


to createH [num_hawks] 
create-custom-hawks num_hawks 
[ 
set color red 
set size 1.0 
setxy random-xcor random-ycor 
] 


end 


to createD [num_doves] 
create-custom-doves num_doves 
[ 
set color white 
set size 1.0 
setxy random-xcor random-ycor 
] 


end 
to fight 


ifelse (is-hawk? self) 
[ 
if ((count other-hawks-here 
[set energy (energy + 0.5 
if ((count other-hawks-here 
[set energy (energy + V)] 


1) and (count other-doves-here 
(v - c))] 


0) and (count other-doves-here 


* 


] 
[ 


if ((count other-hawks-here 


0) and (count other-doves-here 


0) 


1)) 


1)) 


B POGRAM CODE 


[set energy (energy + V / 2)] 
] 
¡set pcolor-of patch-here black 


end 


to move 3; turtle procedure 
rt random-float 50 - random-float 50 
fd 1 
set energy (energy - decrement) 

end 


to reproduce 
if energy > reproduce_limit 


set energy (energy / 2) 
hatch 1 [rt random 360 fd 1] 
] 
if energy < 0 
[die] 


end 


to do-plot 
set-current-plot "ratio" 
set-current-plot-pen "ratio" 
if any? turtles 
[plot count hawks / count turtles] 


end 
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